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(57) Abstract 

A process is defined for the rapid definition of new animation parameters for proprietary Tenderers. The process accommodates the 
peculiarities of proprietary models. In a first step, a proprietary model is animated in a standard modeler and the animated models are 
saved as VRML files. A converter is used to extract the meaning of a newly defined animation parameter by comparing two or more of the 
VRML files. Thus, the output of this process is the model and a table describing the new animation parameter. This information is read 
by the renderer and used whenever the animation parameter is required. Hie process can easily be used to generate new shapes from the 
original model. 
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METHOD FOR DEFINING ANIMATION 
PARAMETERS FOR AN ANIMATION DEFINITION INTERFACE 

This application claims priority to Provisional 
Application Serial No. 60/041,732 filed March 27, 1997 
and entitled "An Animation Definition Interface" and 
Provisional Application Serial No. 60/042,799 filed 
April 7, 1997 and entitled "Let Animals And Furniture 
Speak: Proposal For Extending The Scope OF Face And body 
Animation. " 

The present invention generally relates to the 
field of computer animation and more particularly, is 
directed to a method for defining animation parameters 
for an animation definition interface. 
BACKGROUND OF THE INVENTION 

In recent years, advances in personal computer 
hardware has made it possible to process audio and 
digital images on the desktop. These advances were aided 
by the development of sophisticated formats and 
standards for coding audio-visual information, such as 
movies, video, music and the like, in digital compressed 
formats. Such formats allow large amounts of audio/video 
data to be processed with relatively low cost equipment. 

Under the direction of the International Standards 
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Organization (ISO) and the International Electro- 
Technical Commission (IEC), the "Moving Picture Experts 
Group" ("MPEG") was established to development the MPEG 
suit of standards for coding moving pictures and audio. 
The major advantage of MPEG over other video and audio 
standards is said to be much smaller file size for the 
same quality due to the efficient compression techniques 
employed. 

MPEG encoded video files are typically "played" on 
a PC, Mac or other consumer electronics device using an 
MPEG video player and MPEG audio files are "played" 
using an MPEG audio player. Such players are readily 
available to PC users and the number of MPEG encoded 
video and audio files is steady growing. 

As the need for more demanding and varied encoding 
schemes for video and audio content continue to 
increased, MPEG has been derived into the following 
categories : 

MPEG 1 - Coding of moving pictures 

and associated audio for digital 

storage media at up approximately 

1.5 Mbit/s; 

MPEG 2 - Generic coding of moving 
pictures and associated audio 
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information; and 

MPEG 4 - Coding of audio-visual 

objects. 

Along with the development of MPEG, real-time 
animation on the desk top also has gained popularity in 
recent years. Many real-time animation programs, 
including upcoming MPEG-4 terminals with face and body 
animation capabilities, can be used to run a proprietary 
renderer using a proprietary face or body model. 
Usually, animation of a proprietary model is not 
compatible with MPEG-4 requirements. Furthermore, the 
current implementation and modification of animation 
parameters, like smiles or eyebrow movement, with these 
renderers is cumbersome and time consuming. Thus, there 

is a need in the art for an improved method of 

processing animation parameters. 

SUMMARY OF THE INVENTION 

Accordingly, it is an object of the present 

invention to obviate the above-noted shortcomings and 

disadvantages of real-time animation processes known in 

the prior art. 

Another object of the present invention is to 
provide an animation definition interface for an MPEG 
renderer which can be easily and inexpensively 
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implemented allowing for animation of arbitrary 
downloaded models in the MPEG-4 terminal. 

It is another object of the present invention to 
provide an inexpensive and easily followed method for 
real-time animation. 

The above mentioned disadvantages of prior art 
real-time animation with MPEG compliant renderers can be 
eliminated or substantially reduced by the present 
invention. In accordance with the invention, a method is 
provided that allows the rapid definition of animation 
parameters for proprietary models downloaded from an 
MPEG-4 encoder and their inclusion into proprietary 
real-time rendering software that might also be MPEG-4 
compliant. The proprietary model can be read into any 
commercially available modeler. The modeler is then used 
to define the behavior of different animation 
parameters. For each animation parameter, a modified 
model is created and stored. Model analysis software, 
known as an animation definition interface, compares the 
original model with the animated model and extracts the 
essential animation parameters which represents the 
differences between the original and modified models. 
These parameters are stored in tables and are used by 
the real-time animation program to generate the desired 
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expressions. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The novel features of the present invention are set 
out with particularity in the appended claims, but the 
invention will be understood more fully and clearly from 
the following detailed description of the invention as " 
set forth in the accompanying drawings, in which: 

Figures 1 illustrates how the animation definition 
interface of the present invention is integrated with a 
modeler and renderer; 

Figure 2 illustrates the piece-wise linear 
approximation of a complex deformation applied to a 
vertex of the uniform model; and 

Figure 3 illustrates the use of MPEG-4 for 
animation of computer graphics heads by synthetic speech 
and animation parameters. 



DESCRIPTION OF THE PREFERRED EMBODIMENT 

The MPEG-4 standard described above strives to 
define a standardized interface to allow animation of 
face and body models within an MPEG-4 terminal. Due to 
the rapid advances in computer graphics hardware, it is 
not foreseen that MPEG-4 will standardize face and body 
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models. Instead, face and body definition parameters 
PFDP", "BDP") are defined for specifying the shape and 
surface of a model. For the animation of the models, 
face and body animation parameters ("FAP", "BAP") are 
standardized. 

These animation parameters include low-level 
parameters like "move left eyebrow up'' and "tongue roll" 
as well as high-level parameters like "smile". Assuming 
that different terminals allow for models with different 
degrees of complexity, a process is required that allows 
the rapid development of models suited for animation. 
The use of standardized file format like Virtual Reality 
Modeling Language ("VRML") allow the use of commonly 
available modeling software (modelers), like COSMO 3D or 
PowerAnimator, to design animations. However, formats 
like VRML 1, VRML 2, and Openlnventor only support the 
description of rigid objects. 

VRML was conceived in the early part of 1994 and 
was developed in response to a need for a three 
dimensional graphical visualization standard. VRML 1.0 
was the first version of the standard and provided basic 
support for describing three dimensional objects such as 
spheres, planes, cubes cylinders, cones and the like. 
Version 2.0 of the VRML standard built on the 
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progress of Version 1.0 by adding such capabilities as 
object behavior. 

Face and body animation requires flexible 
deformation. Such a feature is not currently 
conveniently supported by Openlnventor or VRML 2. 
Accordingly, real-time renders which are designed to 
read and write VRML or Openlnventor files must be 
modified in order to accommodate animation. In addition, 
convenient editors for defining the animation 
capabilities are not known in the prior art. 

In accordance with the present invention, an 
interface between a modeler, e.g., Alias/Wavefront Power 
Animator, and real-time animation software is provided. 
The present invention allows for the rapid definition, 
modification and implementation of animation parameters. 

Since the interface reads VRML files from the modeler, 
it is independent of the modeler. The interface writes 
a VRML file and one accompanying table for each defined 
animation parameter thus making this information easily 
integrate able into proprietary renderers. 

The interface of the invention takes as its input 
several VRML files describing static models with a 
topology appropriate for the renderer. Figure 1 
illustrated how the system is integrated with the 
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modeler and the renderer. The model of the Tenderer is 
exported as a VRML file and read into the modeler. in 
order to design the behavior of the model for one 
animation parameter, the model is deformed using the 
tools of the modeler. Usually, restrictions on the 
topology of the model exist. For simplicity, it is 
assumed that the model is deformed only by moving 
relevant vertices and not by changing its topology. The 
modeler exports the deformed model as a VRML file. 

The Animation Definition Interface (DADID) compares 
the output of the modeler with its input, i.e., the model 
exported from the renderer. By comparing vertex 
positions of the two models, the vertices affected by 
the newly designed animation parameter can be 
identified. The ADI computes for each affected vertex a 
3D displacement vector defining the deformation and 
exports this information in a table. 

The renderer reads the VRML file of the model and 
the table in order to determine the definition of the 
new animation parameter. The renderer can now use the 
newly defined animation as required by the animation 
parameters. 

Most of the newly available graphics boards for PCs 
and workstations support rendering based on the OpenGL 
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engine. The VRML 2 file format is based on Openlnventor 
that itself is based on OpenGl. Thus, it is essential 
to enable real-time deformations of models rendered on 
an OpenGl engine. Use of a scene graph usually does not 
allow the movement of parts of an object. Therefore, 
the vertex positions of the scene graph are updated 
through the animation parameters as defined in the 
table. Only the vertex coordinates of the scene graph 
are updated thus allowing one to take full advantage of 
the OpenGl rendering engine speed for global motions, 
lighting texture mapping, etc. 

The conversion process described above allows the 
renderer to only create deformations of moving vertices 
along the defined 3D displacement vector. While this 
might be sufficient for simple actions like move "left 
eye brow up", complex motions like "smile" or "tongue 
roll up" can not be sufficiently modeled by linearly 
moving vertices. Thus, in accordance with the present 
invention, several VRML files are created for different 
phases of the animation or values of the animation 
parameter! Such files allow for a piece-wise linear 
approximation of complex deformations. This process is 
illustrated in Figure 2. 

The following table shows an example of an 
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animation parameter ("AP") definition for 2 vertices, 
where the 3-D motion is defined in 3 intervals. The 
parameter AP is positive and in Interval 1 is valid for 
0 < AP < = 0.33, in Interval 2 is valid for 0.33 < AP <= 
0.66 and in Interval 3 is valid for 0.66 < AP <= 1.0. 



Vertex No. 


1" Interval 


2 nd Interval 


3 rd Interval 


5 


(0.1, 0.7, -5) 


(0, 2, 4) 


(0, 0.1, -0.1) 


13 


(3 ,4, -2) 


(3, 2, 4) 


(-1, -2, -3) 



Given AP = 0.3, vertex 5 would be displaced by: 

0.3 x (0.1, 0.7, -5) T . 
For AP = 0.6, vertex 5 would be displaced by: 

0.33 x {0.1, 0.7, -5) T + (0.6 - 0.33) x (0, 2, 

4) T . 



The above approach can be extended in order to 
allow positive and negative animation parameters as well 
as unequal intervals for defining the motion of a 
vertex. Other extensions include replacing the linear 
approximation of the vertex motion by general functions 
like polynomials. These functions could be different for 
each animation parameter. A further generalization is to 
make these functions dependent on more than one 
animation parameter. 
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For a smile, writing three files with "smile=0.3", 
"smile=0.7" and "smile=1.0" are sufficient to allow for 
a subjectively pleasant piece-wise linear approximation 
of this relatively complex deformation. 

The above outlined procedure can be used to define 
the entire set of MPEG-4 FAPs for a proprietary face 
animation renderer. The model is an extension of 
Parke's model. Applicants have found that FAPs 
integrate nicely with the model's talking capability 
controlled by a TTS system. 

Animated sequences using different personalities 
also are illustrated in Figure 3. 

Arbitrary head models in VRML and animation tables 
can be created in two ways, a natural for personalized 
head models and a synthetic one. 

In the natural approach, a VRML model of a personds 
head is created by, for example, using cyberscan data. 
The animation table is generated by image analysis. 
Images of the person are taken in a neutral state and 
for each facial expression corresponding to the 
different animation parameters. The method described 
above could be used to calculate the displacement 
vectors for IndexedFaceSet nodes. Applying this 
algorithm for different intensities of the person's 
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expression improves the realism of the facial movements 
during animation. An animation system which downloads a 
person' s data obtained in this way represents a new 
architecture for a primitive model-based decoder. 

An example of how the present invention may be 
implemented will now be described. 



ASCII 


Specific 


:tion - FDP 


000 


000 


exposedField 


SFNode 


featurePointsCoord 


NULL 


001 


001 


exposedField 


SFNode 


textureCoord4 Feature 
Points 


NULL 


010 


010 


exposedField 


SFNode 


calibrationMesh 


NULL 


011 


011 


exposedField 


SFNode 


faceTexture 


NULL 


100 


100 


exposedField 


MFNode 


animat ionDe f init i onTabl 
es[ 1 


NULL 


101 


101 


exposedField 


SFNode 


faceSceneGraph 


NULL 



The FDP node defines the face model to be used at 
the receiver. Two options can be supported: 

1. calibration information is downloaded, so that 
the proprietary face of the receiver can be 
configured using facial feature points and 
optionally a 3D mesh or texture; and 

2 a face model is downloaded with the animation 
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definition of the Facial Animation Parameters . 

This face model replaces the proprietary face 

model in the receiver. 

Each field may be described as follows: 

featurePointsCoord contains a Coordinate node. 

Specifies feature points for the 
calibration of the proprietary face. 

The coordinates are listed in the 
''point" field in the Coordinate node 
in the prescribed order, that a 
feature point with a lower label is 
listed before a feature point with a 
higher label (e.g. feature point 
3.14 before feature point 4.1). 

contains a TextureCoordinate node. 
Specifies the texture coordinates 
for the feature points. 

contains an IndexedFaceSet node. 
Specifies a 3D mesh for the 
calibration of the proprietary face 
model. All fields in the 
IndexedFaceSet node can be used as 
calibration information. 

contains an ImageTexture or 
PixelTexture node. Specifies 
texture to be applied on the 
proprietary face model. 

animationDef inition 

tables contains AnimationDef initionTable 

nodes. If a face model is 
downloaded, the behavior of FAPs is 
defined in this field. 

faceSceneGraph contains a Group node. Grouping 

node for face model rendered in the 
compositor. Can also be used to 
download a face model: in this case 
the effect of Facial Animation 
Parameters is defined in the 



textureCoord4- 
Feature Points 



calibrationMesh 



f aceTexture 
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"animationDef initionTables" field. 

Other ASCII specifications are set forth in the 
tables below: 



?? AnimationDef initionTable 


00 


field SFInt fapID 1 


1 


68 


01 


field SFInt highLevelSelect 1 


1 


64 


10 


0 exposedfield Mfnode table {] NULL 









?? 


AnimationDef Transform 




00 


field 


?? nodelDTransform 




01 


field 


SFString fieldID 


na 


10 


field 


GenericSF f ieldValue 


na 





?? 


AnimationDef IndexedFaceSet 


00 


field 


?? nodelDIndexedFaceSet 


01 


field 


MFInt32 intervalBorders [ ] 


10 


field 


MFint32 coordlndex [] 


11 


field 


MFVec3f displacements [] 



While the present invention has been described with 
respect to a preferred embodiment, the invention is not 
limited thereto. In general, this version is applicable 
when defining animation parameters defining flexible 
deformation of computer graphic models. The scope of the 
invention is limited only by the attached claims. 
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CLAIMS 

1. A method for defining a set of computer animation 
parameters for an object to be animated electronically, 
wherein said animation is achieved by electronically 
altering at least a portion of said object in a 
controlled manner, said method comprising the steps of: 

obtaining an electronic reference model of said 
object to be animated; 

altering said reference model to form a modified 
model corresponding to a first animation parameter; 

determining the physical differences between said 
reference model and said modified model and storing said 
differences as said first animation parameter; 

repeating said altering and said determining steps 
for each of said animation parameters to be defined; and 

providing said stored parameters to a rendering 
device for generation of said animation in accordance 
with said stored parameters. 

2. The method of claim 1 further including the 
definition of an object as a scenegraph and the 
definition of high level animation parameters to allow 
rigid and flexible deformation. 

3. The method of claim 1 further including the step of 
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defining a major animation parameter, said step includes 
the sub-steps of: 

defining a plurality of associated animation 
parameters representing an animation sequence; and 

storing said plurality of associated animation 
parameters as said major parameter. 

4. The method of claim 1, wherein said object includes 
characteristics of a human being and said method further 
includes the step of defining a major animation 
parameter representing a human expression, said step 
includes the sub-steps of: 

defining a plurality of animation parameters 
representing said human expression; and 

storing said plurality of animation parameters as 
said major parameter. 

5. The method of claim 1, wherein said animation is 
described by spline functions. 

6. The method of claim 1, wherein said animation is 
described by rational polynomial functions . 

7. The method of claim 1, wherein, said animation is 
described by general functions. 

8. The method of claim 1, wherein said determining step 
includes the sub-step of comparing the vertex positions 
of said modified model to the vertex positions of said 
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reference model. 

9. The method of claim 1, wherein said determining step 
includes the step of approximating the differences 
between said reference model and said modified model and 
storing said differences as said first animation 
parameters . 
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